If you want to utilize Entity Framework in a basic .NET Aspire application, adding a project to contain the entity models, context, and Entity Framework references and supporting a database engine container is a recommended place to get started. I suspect this guidance may be refined as .NET Aspire evolves.
.NET Aspire is a way to define your distributed system, and then IaC & Deployment tooling have to pick up the slack getting your definition out there, is this a good thing?